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Yelp's  Mission: 
Connecting  people  with  great 
local  businesses. 
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Yelp  Stats 

As  of  Q1  2016 


Macro  Threat  Intelligence 


Companies  that  specialize  in 
endpoint  security  look  for 
patterns  across  their 
customer  base,  then  apply 
those  signatures  or 
heuristics  to  your 
environment. 

This  is  a good  thing. 
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Micro  Threat  Intelligence 


Analysts  dedicated  to  your 
environment  know  what's 
normal  (and  what's  not.) 

This  is  also  a good  thing. 
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Which  one  do 
you  need? 

BOTH. 
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A Short,  Simplistic  History  of  Malware 


In  the  beginning  (1 988)  people  wrote  malware  to  prove  it 
could  be  done. 


Then  they  wrote  malware  for  lulz. 


Now,  it's  for  money  or 
country. 
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Modern  attackers  use  camouflage 


Tweak  commodity 
grade  malware  to 
be  unique  enough 
to  avoid  AV 


Use  your  own  tools 
against  you 
(Powershell,  SSH)  to 
limit  their  footprint 


Tshjjeb 
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Use  macro  and  micro  perspectives 
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"Our  web  proxy  says  that 
activity  is  suspicious,  but  what' 
s normal  for  our 
environment?" 


"AV  didn't  flag  this  executable, 
but  it's  only  installed  on  one  of 
our  machines.  What  does 
VirusTotal  say?" 

Surreal  Landscape  Photography  by  Randy 


What  does  our  DNS  resolver  do? 


• Blocks  access  to  ${bad}  by  blocking  the  resolution  of 
suspect  domain  names 

• Allows  us  to  pull  the  logs  about  what  our  endpoints 
resolved  when  via  the  API 
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Spike  in  Blocked  DNS  example 


DNS  resolver  knows  a lot 
about  which  sites  are  bad. 
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STAND  BACK 


I’M  GDING  TD  TRY 


Start  by  defining  two  groups 


A group  of  non-infected 
machines  (your  control 
group) 
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A group  of  machines  you 
know/suspect  are  infected. 


Collect  (and  alert)  on  DNS  data  in  ELK 


* » elastic  + 


kibana 


Sample  record 


"_index":  "logstash-| 

"-type":  "opendns", 

"_id" : " AVShllVVDQqXJoZUJ unf " , 

"_score":  null, 

"_source":  { 

"@ingestionTime" : "2016-05-llT18:44: 19.268159Z" 
"event_time" : "2016-05-11T18 : 29 : 55+00 : 00" , 
"opendns_data" : { 

"domain":  "cdn.freefaits.com. ", 

"external_ip" : 

" response_code" : "NOERROR" , 

"internal_ip" : 

"query_type" : "1  (A)", 

"action":  "Allowed" , 

"identities": 

"categories" : 

"most_granular_identity" : | 

}, 
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Researched  37  potential  infections 
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notjnfected  not  many  red  flag  YELP- 

notjnfected  guilt  by  associate  YELP- 


Educated  guess  Notes 

infected  lots  of  illumationes[.)net  domains,  like  ***** 

infected  http://research.zscaler.com/2015/09/compromised-wordpress-campaign-spyware.hi 

infected  lots  of  illumationes[.]net  domains,  like  ***** 


Initial  research  may  be  manual  and  time-consuming. 
It's  okay  to  start  with  a small  group  and  test  more 
later.  Just  watch  for  over-fitting. 
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infected  redacted(.]redacted[.]com  and  hI.]redacted(.]com  have  redflags  on  VT 
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infected  www[.]colbridge[.]com  is  a malware  site  according  to  both  VT  and  OpenDNS 

YELP-* 

infected  looks  like  songwriter’s  website  was  compromised  and  used  - VT  sample  7a939e52439cb364cad0a0f 
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What  differs  between  these  groups? 
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big  data  dashboard 

i 

:|i§  S)  A 

"After  careful  consideration  of  all  437  charts,  graphs,  and  metrics, 
I've  decided  to  throw  up  my  hands,  hit  the  liquor  store, 
and  get  snockered.  Who's  with  me?!" 
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How  about  some  data  visualization? 


Sparklines++ 
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Hypothesis:  A peak  n blocked  DNS 
lookups  followed  by  low-level  activity 
indicates  infection. 
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Fun  with  Excel  formulas 
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^ Would  my  heuristic 
have  called  this 
machine  infected? 


Did  I call  an 
infected  machine 
infected?  A non- 
infected  machine 
infected? 
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How  to  judge  an  alert  by  the  numbers 


True  positive: 
how  many 
infected 

machines  did  we 
call  infected? 
True  negative: 
how  many  clean 
machines  did  we 
call  clean? 


False  positive: 
how  many  clean 
machines  did  we 
call  infected? 
False  negative: 
how  many 
infected 

machines  did  we 
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Why  these  numbers  matter 


True 

positive 


False 

positive 


True 
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Unfortunately  in  general 


• • • 


Sensitivity 


Decreasing  the  false 
negative  rate  will  increase 
the  false  positive  rate. 

Finding  more  infections 
(increasing  sensitivity) 
means  triggering  more 
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Find  your  organization's  balance 


• How  critical  is  the  data 
you're  protecting? 
Nuclear  missile  launch 
codes?  T rade  secrets? 
Inventory? 

• How  many  people  do 
you  have  for  incident 
response?  Do  you 
automate? 
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After  the  hypothesis,  experiment 


What  if  we  say  an  initial  peak  of  at  least  5 
blocked  DNS  resolutions  (P=5),  plus  at 
least  three  subsequent  days  with  3 or 
more  blocked  resolutions  (D=3)? 
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New  hypothesis 

Original  hypothesis:  P=5,  D=3 


New  hypothesis:  P=10,  D=3 

New  new  hypothesis:  P=15, 
D=3 

New  new  new  hypothesis: 
P=<redacted>,  D=<redacted> 
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How  educated  were  those  guesses? 

Of  the  21  machines  I labeled  as  suspicious: 

• 1 2 were  not  infected  probably 

• 2 were  infected  (confirmed  via  other  means) 

• 6 were  not  examined  any  further 


If  we  feed  those  results  back  into  our 
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False  positive  rate  too  high? 
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Naw,  automate 
some  analysis 
with  osquery. 


Osquery 


We  use  osquery  to  periodically 
collect  diagnostic  information  like: 

• installed  programs 

• kernel  extensions 

• programs  with  listening  ports 

• etc 


MM] 
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Osquery  + ELK  + Python 


OSXcollector  - now  remote! 


We  use  our  package 
management  tool  to 
install  and  run 
OSXcollector  remotely. 
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DNS  resolver's 
intelligence  (macro) 
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Infections  AV/IDS 
didn't  find 


Local  peaks  in  blocked 
resolutions  (micro) 


SCIENCE 


Machines  to  be 
examined 


Sample  alert  triggered 

('2015-10-08',  29,  {'bnk[.]annoysalbania[.]com.':  8,  'exm[.]enticingsuperpower[.]com.':  21}) 
('2015-10-09',  43,  {'fbs[.]hearkenperturbation[.]com.':  7,  'exm[.]enticingsuperpower[.]com.':  36}) 
('201 5-1 0-1 1 ',  1 6,  {'ylu[.]lovebirdsalphabetic[.]com.':  1 1 , 'omq[.]relievingdungeons[.]com.':  5}) 
('201 5-1 0-1 2',  1 1 , {'isi[.]envelopspunnet[.]com.':  1 1 }) 

('2015-10-13',  10,  {'isi[.]envelopspunnet[.]com.':  9,  'dlo[.]beatingsmiler[.]com.':  1}) 

('2015-10-14',  2,  {'xwm[.]solemnpertness[.]com.':  2}) 

('2015-10-15',  35,  {'hgh[.]filletoutdated[.]com.':  22,  'xwm[.]solemnpertness[.]com.':  13}) 
('2015-10-16',  21,  {'hgh[.]filletoutdated[.]com.':  21}) 

('201 5-1 0-1 7',  76,  {'hgh[.]filletoutdated[.]com.':  45,  'swf[.]chequebooksbruising[.]com.':  31 }) 
('2015-10-18',  32,  {'ils[.]infernalbrazing[.]com.':  10,  'swf[.]chequebooksbruising[.]com.':  22}) 

('201 5-1 0-1 9',  44,  {'ils[.]infernalbrazing[.]com.':  41 , 'elu[.]proletarianisationsinus[Jcom.':  3}) 
('2015-10-20',  63,  {'hqi[.]slugsqueals[.]com.':  51,  'elu[.]proletarianisationsinus[. 
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Tell  me  more  . . . 

►Hvirustotal 

SHA256:  Ol5a8bad618a3e2454587c9530ce6aae95fd4c407300bb30fb76043853{ 

Detection  ratio:  12/56 

Analysis  date:  2015-10-09 19:12:06  UTC  ( 7 months,  1 week  ago ) 

[■Analysis  <H  File  detail  © Additional  information  •Comments  Q <JVot 
Antivirus  Result 


AhnLab-V3 

Arcabit 

Avira  (no  cloud) 


Gen:Variant.Kazy.580380 
Trojan/Win32.Crossrider 
Trojan.Kazy.D8DB1  C 
TR/Kazy. 151 3679 
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"@ingestionTime" : "2015-10-05T23 : 54:032", 

"columns":  { 

"bundle_identifier" : "com. PCvark. Advanced-Mac-Cleaner", 
"name":  "Advanced  Mac  Cleaner. app", 

"path":  "/Applications/Advanced  Mac  Cleaner. app" 

h 

"filter_result" : "blacklisted", 

"kibana_link" : "https ://^^^^^^^^^^^^^^^/#/dashboard/Os 


20151009 

20151009 

20151009 

20151009 


20151009 


Another  sample  alert  triggered 


(' 2016-01-09 ' , 21,  Counter({ ' standout [. ]tv[ : 21})) 

('2016-01-10',  6,  Counter({ ' ads2[ . ]contentabc[ . ]com[ . ] ' : 6})) 

('2016-01-11',  5,  Counter({ ' bttrack[ . ]com[ . ] ' : 2,  'cdn[ . ]bttrack[ . ]com[ . ] ' : 2, 
'94982c5b634975e50103ce96082d2827[.]adsk2[.]co[.] ' : 1})) 

( 1 2016-01-12 ' , 20,  Counter({ ' ads2[ . ]contentabc[ . ]com[ . ] ' : 8,  ' loadm[ . ]exelator[ . ]com 
5,  ' standout [. ]tv[ .]' : 3,  ' loadus[ . ]exelator[ . ]com[ . ] ' : 2,  ' secure-au[ . ] 
imnworldwide[ . ]com[ . ] ' : 1,  ' 1049theeagle[ . ]com[ . ] - Hosts  malware':  1})) 
('2016-01-13',  47,  Counter({ ' ads2[ . ]contentabc[ . ]com[ . ] ' : 22,  ' www[ . ]4chan[ . ]org 
[.]':  14,  'sys[ . ]4chan[ . ]org[ . ] ' : 8,  '4chan[ . ]org[ . ] ' : 2,  ' cdn [ . ]directrev[ . ] com 
1})) 

('2016-01-14',  2,  Counter({ ' ads2[ . ]contentabc[ . ]com[ . ] ' : 2})) 
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Tell  me  more 


• • • 


Osquery  comparison  script  found  ZipCIoud.app  installed: 

00025  apps  named  ZipCIoud.app*  found 

OSXcollector  found  signs  of  Genieo: 

- chrome  local_storage 

key:  "CLOUDFLARE : : http://static.genieo.com/download/download_genieo2.js" 

value:  "{"url" : "http: //static .genieo.com/download/download_genieo2. js" , "contents" : "var  _gaq=_gaq | | [] ; 

(f unction ( ) \n{var  popupStyles={ ' default 1 : {path : ' popups/popup_mac_style . html ' , width : 1 100% 1 height : 

' 100%' j margin : '0' }, 'with_logo' : {path: ' popups/popup_with_logo. html ' , width: ' 100%' , height: ' 100%' , margin : '0' }} 
\n_genieo=_genieo | | {};_genieo[ ' style' ]=_genieo[ ' style ' ] | | 'default ' ;var  partner=_genieo[ ' partne 
r ' ] | | 'genieo' ;var  campaign=_genieo[ ' campaign ' ] | | ' ' ;var  hidden=_genieo[ ' hidden ' ] | | false; var" 

*ZipCloud  is  often  bundled  with  undesirables 


any. 
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Are  you  a DFIR  minion? 


Join  DFIR_MNions! 


Email  dfirmnions@gmail.com 
and  tell  me:  who  you  are,  why 
you  want  to  join,  if  anyone  can 
vouch  for  you. 

Meetings  are  informal  (for  now) 


pi]  Iff 


@PwnieFan 


yelp3> 

fb.com/YelpEngineers 

@YelpEngineering 

engineeringblog.yelp.com 


github.com/yelp 


